Method and system for quantifying technical skill

ABSTRACT

A system and method for quantifying clinical skill of a user, comprising: collecting data relating to a surgical task done by a user using a surgical device; comparing the data for the surgical task to other data for another similar surgical task; quantifying the clinical skill of the user based on the comparing of the data for the surgical task to the other data for the other similar surgical task; outputting the clinical skill of the user.

This application claims priority to patent application 61/162,007, filedMar. 20, 2009, entitled “Method for Automatically Evaluating Skill forMotion Training”, which is herein incorporated by reference.

This invention was made with government support under 0534359,EEC9731478 and 0205348, awarded by the NSF, as well as an award by theNSF Graduate Research Fellowship Program. The government has certainrights in the invention.

FIELD OF THE INVENTION

The invention is in the field of training, and in one embodiment,surgical training.

BACKGROUND OF THE INVENTION

Virtual training systems have gained increasing acceptance andsophistication in recent years. However, inadequate training can lead toa higher incidence of mistakes. Thus, clinicians desire a more objectivemethod for quantifying clinical technical skill.

Various systems that involve a human-machine interface, includingvirtual systems, can involve human motions that are random in nature. Aperson performing a repeatable task multiple times often generatesdifferent motion measurements (e.g., forces, velocities, positions,etc.) despite the fact that the measurements represent the same taskperformed with the same level of skill. Thus, skill modeling shoulduncover and measure the underlying characteristics of skill hidden inmeasurable motion data.

One example of such a system that includes a human-machine interface isa teleoperated robotic surgical system, such as the da Vinci® SurgicalSystem commercialized by Intuitive Surgical, Inc. A skilled operator mayperform a particular task many times when using a teleoperated roboticsurgical system, even though the operator exhibits many small motioncharacteristic variations among the many task performances. And, anoperator with a less proficient skill level will often exhibit motioncharacteristics when performing the particular task that aresignificantly different from the skilled operator's motioncharacteristics for the task.

What is desired is a way to identify how an unskilled or lesser skilledoperator's motion characteristics compare with a skilled operator'smotion characteristics so that the unskilled or lesser skilledoperator's task proficiency can be objectively quantified. What is alsodesired is a way to provide an objective quantification of an operator'sskill level that can be used to help train the operator to perform at ahigher skill level. In particular, it is desirable to objectivelyquantify particular surgical task performances of a surgeon who islearning to use a telerobotic surgical system, and then to use the taskperformance information to help the surgeon achieve a more proficientperformance level.

BRIEF DESCRIPTION OF THE FIGURES

FIGS. 1, 9, and 13-14 illustrate details related to a surgical systemfor quantifying technical skill, according to several embodiments.

FIGS. 2-8 and 12 illustrate examples of quantifying technical skill,according to multiple embodiments.

FIGS. 10-11 illustrate a method for quantifying technical skill,according to several embodiments.

DESCRIPTION OF EMBODIMENTS OF THE INVENTION

A system and method are provided for quantifying technical skill. Datacan be collected for a surgical task that a user performs. The data canthen be compared to other data for the same surgical task. The level ofexpertise of the user can then be determined based on the comparing, andthe clinical skill of the user can be quantified.

In some embodiments, data indicating how a skilled user performs asurgical task can be collected, and this data can be compared tocollected data indicating how a second user performs the surgical taskso as to determine the second user's clinical skill. In someembodiments, the collected data indicating how a skilled user performs asurgical task can be used to train the second user.

System for Quantifying Technical Skill

FIG. 1 illustrates a surgical system 100, according to one embodiment.In system 100, data is collected and archived. In one embodiment, thesurgical system 100 can be the da Vinci® Surgical System, commercializedby Intuitive Surgical, Inc. Additional information on the da Vinci®Surgical. System can be found in, e.g., U.S. Pat. No. 6,441,577 (filedApr. 3, 2001; disclosing “Manipulator Positioning Linkage for RoboticSurgery”) and 7,155,315 (filed Dec. 12, 2005; disclosing “CameraReferenced Control in a Minimally Invasive Surgical Apparatus), both ofwhich are herein incorporated by reference. Although the da Vinci®Surgical System can be used in one embodiment, those of ordinary skillin the art will see that any surgical system can be used. Those ofordinary skill in the art will also see that there are other ways tocollect data, and that embodiments of the invention can be in manyfields other than surgery, including but not limited to: rehabilitation,driving, and/or operating machinery.

In one embodiment, the surgical system 100 can include a surgeon'sconsole 105, a vision cart 125, and a patient side cart 110. These mainsystem 100 components may be interconnected in various ways, such as byelectrical or optical cabling, or by wireless connections. Electronicdata processing necessary to operate system 100 may be centralized inone of the main components, or it may be distributed among two or moreof the main components (a reference to an electronic data processor, acomputer, or a similar term, therefore, can include one or more actualhardware, firmware, or software components that may be used to produce aparticular computational result).

The patient side cart 110 can include one or more robotic manipulatorsand one or more movable surgical instrument components associated withsuch manipulators, such as the ones illustrated in FIG. 13. FIG. 13illustrates various possible kinematic components and their associatedmovements (e.g., degrees of freedom, which may be variously defined aspitch, yaw, roll, insertion/withdrawal, grip, and the like) and alsoillustrative joints that may be associated with degrees of freedom forthese components. FIG. 14 illustrates possible parameters (data points)relating to these degrees of freedom, as well as other system components(e.g., kinematic parameters such as joint position and velocity,Cartesian position and velocity, rotation matrix values, etc. for themaster manipulators; joint position and velocity, Cartesian position ofthe remote center of motion, rotation matrix values, set up jointvalues, etc. for the patient side cart; various servo times, buttonpositions, etc., at various places on the system; etc.). These dataparameters can be used when measuring a surgeon's movements, which maybe characterized by surgeme and dexeme motions that are described inmore detail below.

As illustrated by system 100, the surgical system may include anapplication programming interface (API), which may be accessed via anEthernet connection on, e.g., an interface 115 on surgeon's console 105or on another system component. Various system 100 parameters, such asthose identified with reference to FIG. 14, may be monitored andrecorded (stored, archived, etc.) via the API.

Video data collected by an endoscopic imaging system mounted on patientside cart 110 may be processed through vision cart 125 and output to thesurgeon at surgeon's console 105. The video data may be stereoscopic(e.g., left and right eye channels, so as to give the illusion of depthin an apparent three-dimensional (3-D) image) or it may be monoscopic.The video data may be accessed via one or more video output ports insystem 100, such as video output connectors located on interface 115.The accessed video data may be recorded, and the video data recordingmay be synchronized with data output via the API so that systemparameters being monitored and video data may be recorded and stored assynchronized with one another.

As shown in FIG. 1, system 100 includes a computer 135, which may be aplatform separate from and connected to one or more of the other system100 components, or which may be integral with one or more of the othersystem 100 components. A quantifying skill computer application 130 canbe stored in a memory to be accessed and executed by computer 135.

FIG. 9 illustrates details of the quantifying skill computer application130, which can include a user interface 910, a comparing module 915, amodeling module 905, a teaching module 920, and a segmenting module 925.The user interface 910 can be used to interact with the user. Forexample, the user interface 910 can display the motions and sub-motionsthat were tracked for a test, and also indicate which group that textwas classified as, as well as disclosing the data behind thatclassification. The segmenting module 925 can be used to segment datafrom a procedure into surgemes and dexemes. The formula used to segmentthe data is described in more detail below. The comparing module 915 canbe utilized to compare the data from a test user with data representingexpert data, intermediate data, or novice data (or any level ofexpertise), and determine which level the test user should be designatedas, based on the test user's movement data. The modeling module 905 canmodel movements of a particular skill level (e.g., an expert surgeon).For example, the modeling module 905 can take data that representsmovements of an expert user and model those movements. The teachingmodule 920 can be utilized to teach a user how to do a particular taskor sub-task. For example, the teaching module 920 can utilize the datamodeling the movements of an expert and use that modeled data to train auser. In some embodiments, the data modeling the movements of an expertcan be obtained from the modeling module 905.

Collected data can be encrypted and transferred to an attached portablecartridge (e.g., coupled to computer 135; not shown) using a cartridgedrive at the end of a data collection session. Many recorded procedurescarried out by one or more persons can be stored on the cartridge. Thedata from the cartridge can be uploaded to a secure repository (e.g.,via a network or internetwork, such as the Internet), or the data fromthe cartridge drive can be physically sent to another system for storageand/or analysis. Alternatively, the collected data can be transferredfrom computer 135 directly via network or internetwork to a computer atanother location for storage and/or analysis.

An anonymized list of users that use the surgical system 100 can bemaintained, and each user can be assigned a unique ID. The collected andarchived data can use the unique ID so that the user can be identifiedonly by the unique ID when doing further analysis.

Archived data can be segmented at various granularity levels for aparticular trial, task, or procedure. For example, the archived data maybe segmented into trial (e.g., procedure level) data, surgeme (e.g.,procedure sub-task level) data, or dexeme (e.g., particular motioncomponent of sub-task level) data. These levels of data, and how theyare utilized, are described in more detail below.

Archived data can be securely stored. In one embodiment, only users orentities participating in the data collection may access the archiveddata.

Method for Quantifying Technical Skill

FIG. 10 illustrates a method for quantifying technical skill. In 1005,data can be gathered from one or more surgical systems that are used toperform surgical procedures. In one embodiment, a telerobotic surgicalsystem such as the da Vinci® Surgical System can be utilized. In 1010,the data is segmented and labeled. In 1015, the segmented data can becompared to other segmented data and analyzed. The analyzed data canthen be utilized to quantify the skill of the users of the surgicalsystem. Details related to these elements are described in more detailbelow.

Gather Data

Still referring to FIG. 10, in 1005 data can be gathered from one ormore surgical systems that one or more surgeons use to perform surgicalprocedures. Thus, for example, motion data can be gathered from surgeonswho have different expertise levels as the surgeons perform surgicaltasks using the one or more surgical systems. For example, in oneembodiment, a telerobotic surgical system can be used to perform a trial(e.g., procedure) that involves a suturing task (e.g., surgical joiningof two surfaces). Data can be collected using the telerobotic surgicalsystem. The data can comprise multiple positions, rotation angles, andvelocities of the surgeon console master manipulators and/or the patientside manipulators of the telerobotic surgical system. The gathered datamay also include video data collected from the surgical system duringthe trial or a portion of the trial, as described above.

Segment and/or Label Data

Still referring to FIG. 10, in 1010 the trial data can be segmentedand/or labeled.

FIG. 2 illustrates various levels that can be used to segment (break up)a procedure, according to one embodiment. As noted above, recorded datacan be segmented into trial (e.g., procedure) data, task data, surgeme(e.g., sub-task) data, or dexeme (e.g., motion of sub-task) data. Skillevaluation and training can be done at each level. P1 can be the trialor procedure level (e.g., radical prostatectomy, hysterectomy, mitralvalve repair). T1 and T2 are illustrative of various task levels (e.g.,suturing), which are tasks that need to be done in the procedure. S1-S6are illustrative of surgeme levels (e.g., needle pulling), which aresub-tasks needed for a task. As shown in FIG. 2, for example, task T1 issegmented into surgemes S1-S3, and task T2 is segmented into surgemesS4-S6. M1-M6 are illustrative of various dexeme levels, which are motionelements of a sub-task (dexemes represent small dextrous motions).Dexemes can be used to distinguish temporal sub-gestures of a singlegesture, as well as stylistic variations between samples of the samegesture. For example, some gestures in a suturing task, such asnavigating a needle through the tissue, can be more indicative ofexpertise than other gestures, such as pulling thread. Such fine grainedassessment can lead to better automatic surgical assessment andtraining. As illustrated in FIG. 2, for example, surgeme S2 is segmentedinto dexemes M1, M4, and M2, and surgeme S5 is segmented into dexemesM5, M4, and M3. Thus a particular dexeme may be a component of a singlesurgeme, or it may be a component of two or more surgemes. Likewise, anyrelatively finer grained segment may be a component of only one or morethan one relatively courser grained segment of the next highest level.

FIG. 3 illustrates how various surgemes can be manually segmented andlabeled, according to one embodiment. FIG. 3 illustrates an example ofnine surgemes associated with a suturing task (not necessarily inorder), with their respective labels. The following motion labels areprovided to the nine surgemes: (0) idle position, (1) reach for needle,(2) position needle, (3) insert needle through tissue, (4) transferneedle from left to right hand, (5) move to center with needle in righthand, (6) pull suture with left hand, (7) pull suture with right hand,and (8) orient needle with both hands (the idle state may or may not beconsidered a surgeme; idle time doing nothing may be a characteristicthat is desirable to monitor). In this example, the data is manuallysegmented and labeled. The surgemes can then be manually segmented intodexemes.

In some embodiments, the data can be automatically segmented intosurgemes. The motion data can be automatically segmented by normalizingthe data and projecting it to a lower dimension using lineardiscrimination analysis (LDA). (For more information on LDA, see Fisher,R.: The use of multiple measurements in taxonomic problems. Annals ofEugenics 7 (1936) 179-188.) A Bayes classifier can then decide the mostlikely surgeme present for each data in the lower dimension based onlearned probabilities from training labeled data sets. For moreinformation on how the data can be automatically segmented, see H. Linet al., “Towards Automatic Skill Evaluation: Detection and Segmentationof Robot-Assisted Surgical Motions”, Computer Aided Surgery, September2006, 11(5): 220-230 (2006), which is herein incorporated by reference.

In one embodiment, this automatic classification can be checked foraccuracy. In order to do this, {σ_([i]), i=1, 2, . . . , k} can be usedto denote the surgeme label-sequence of a trial, with σ_([i]) in the set{1, . . . , 11} and k≈20, and [b_(i), e_(i),] the begin and end-time ofσ_([1]), 1≦b_(i)<e_(i)≦T. Note that b₁=1, b_(i)+1=e_(i)+1, e_(k)=T. Asurgeme transcript {{circumflex over (σ)}_([i]). i=1, 2 . . . ,{circumflex over (k)}} and time marks [{circumflex over (b)}_(i), ê_(i)]can be assigned to the test trial.

Determining the accuracy of the automatic segmentation {y₁ . . . y_(T)}as compared to manual segmentation can then be done using the followingformula:

Accuracy of test trial

$\mspace{20mu} {\left\{ {y_{1},\ldots \mspace{14mu},y_{T}} \right\} = {\frac{1}{T}\text{?}{\left( {\sigma_{t} = {\hat{\sigma}}_{t}} \right)}}}$?indicates text missing or illegible when filed

where σ_(t)=σ_([i]) for all tε[b_(i), e_(i)] and {circumflex over(σ)}_(t)={circumflex over (σ)}_([i]) for all tε[{circumflex over(b)}_(i), ê_(i)].

The surgemes can also be automatically segmented using other methods.For example, in another embodiment, the motion data can be automaticallysegmented by normalizing the data and projecting it to a lower dimensionusing linear discrimination analysis (LDA), as described above. Then,the lower dimension data x_(t) can be plugged in the following formulaand run for every possible value for σ (which can represent every typeof way to segment the lower dimension data).

$\begin{matrix}{{{P_{\sigma}\left( {\text{?},\ldots \mspace{14mu},\text{?}} \right)} = {\text{?}{\sum\limits_{s_{b_{i} + 1} \in S_{\sigma}}\mspace{14mu} {\ldots \mspace{14mu} {\sum\limits_{s_{e_{i}} \in S_{\sigma}}{\text{?}{p\left( s_{t} \middle| s_{t - 1} \right)}{\left( {{x_{t};\mu_{s_{t}}},\sum_{s_{t}}} \right)}}}}}}},{\text{?}\text{indicates text missing or illegible when filed}}} & (2)\end{matrix}$

where S_(σ) denotes the hidden states of the model for surgeme σ,p(s|s′) are the transition probabilities between these states, and N(•;μ_(s), Σ_(s)) is a multivariate Gaussian density with mean μ_(s) andcovariance Σ_(s) associated with state sεS_(σ).

The value of σ that gives the maximum value of P is the segmentationthat is used for the surgemes.

The same formula can be used to break up the lower dimension data intodexemes. If we use a Viterbi algorithm to segment the projectedkinematic data with respect to the HMM state-sequences, we get a dexemelevel segmentation of the data. Such dexeme-level segmentation arevaluable for performing dexterity analysis. For more information onViterbi algorithms, see L. Rabiner, “A Tutorial on Hidden Markov Modelsand Selected Applications in Speech Recognition”, IEEE 77(2) (1989)257-286.

A discrete HMM can be represented by λ (=A, B, π), which can include:the state transition probability distribution matrix A=a_(ij), wherea_(ij) is the transition probability of a transition from state i tostate j; the observation symbol probability distribution matrixB=b_(j)(k) where b_(j)(O_(k))=P[o_(t)=v_(k)|q_(t)=j] is the outputprobability of symbol v_(k) being emitted by state j; and the initialconditions of the system π. For more information on HMMs, see L.Rabiner, “A Tutorial on Hidden Markov Models and Selected Applicationsin Speech Recognition”, IEEE 77(2) (1989) 257-286.

FIG. 8 illustrates a 5 state HMM for a particular surgeme correspondingto the act of “inserting needle through the tissue”, according to oneembodiment. Individual dexemes corresponding to HMM states a, b, c, d,and e can be isolated. It can then be determined that certain dexemes(e.g., a, b, c) constitute rotating of the right hand patient-side wristto drive the needle from the entry to the exit. In addition, it can bedetermined that, for example, the dexeme c movement, which correspondsto a sub-gesture where the surgeon hesitates/retracts while pushing theneedle to the exit point, was from mostly novice surgeons.

Compare Data and Quantify Clinical Skill

Referring back to FIG. 10, in 1015, after the trial is segmented and/orlabeled, clinical skill can be quantified by making comparisons betweendata.

The segmented data produced in accordance with 1010 in FIG. 10 can beused to identify the most likely skill model to have produced certainsegmented data. For example, once the data has been segmented into asequence of surgemes or dexemes, this sequence O_(test) can be comparedto various skill level models λ_(e) (expert), λ_(i) (intermediate), andλ_(n) (novice). The skill level of the test data λ_(test) can be labeledexpert, intermediate or novice based on which skill level is closest tothe test data, based on the following distance formula:

${D\left( {\lambda_{s},\lambda_{test}} \right)} = {\frac{1}{T_{test}}{\min \left( {{\xi \left( {\lambda_{i},\lambda_{test}} \right)},{\xi \left( {\lambda_{e},\lambda_{test}} \right)},{\xi \left( {\lambda_{n},\lambda_{test}} \right)}} \right)}}$

where:

ξ(λ_(s),λ_(test))=log P(O _(test)|λ_(test))−log P(O _(test)|λ_(s))

and λ_(s) is the skill model, and T_(test) is the length of theobservation sequence O_(test).

It should be noted that the motion labels can be used to exploreappropriate ways for evaluating the skill of the motions. In addition,the time per task (including the time per surgeme and dexeme) can becompared. In some embodiments, idle motion time at the start and end ofthe trial (motion (0)) does not need to be used for data analysis. Themotions, the timing of the motions, and the sequence of motions executedby the user can be used to make conclusions about the relative skill ofa user that is performing each trial.

For example, FIG. 4 illustrates the difference between the movements ofexperts, intermediates, and novice surgeons. As the surgeon's skillincreases, the graph of his or her movements shows that the movementsbecome more directed. In this example, the expert surgeon (shown asgraphs (a) and (d)) accomplishes a task using fewer movements, whereasthe novice surgeon (shown as graphs (c) and (f)) made more errors duringthe task and thus used extraneous motions and started over. FIG. 4 alsoillustrates that an idle surgeme during a task may represent an error(e.g., dropping a needle), and so may be significant to a skill levelanalysis. Thus an otherwise substantially similar surgeme may beassigned a separate label, or it may be identified as significantbecause of its position in a sequence of surgemes.

FIG. 5 illustrates typical transitions between surgemes during a sampletrial. The transitions between surgemes reveals immediate differences inthe approach taken between experts and novices. Experts can use oneparticular pattern of motions repeatedly throughout the task.Consequently, users who have a relatively higher skill level can createmore directed transition graphs than users who have a relatively lowerskill level. For example, after pushing the needle through simulatedtissue from the target entry point to the target exit point, as shown inthe top portion of FIG. 5, an expert's trials can show the suture ispulled taut with the left tool, and then the needle is handled to theright tool for another round of positioning and insertion (this sequenceis represented as surgemes 6, 4, 2, 3 in the bottom portion of FIG. 5).In contrast, a less experienced surgeon's trials can show the sutureoccasionally being pulled a portion of the way with the left tool withthe right tool then used to pull the suture taut (this sequence isrepresented as surgemes 6, 7, 2, 3 (not shown)). In addition, FIG. 5illustrates that the duration of a sequence of one or more surgemes canbe measured. In one instance in which simulated tissue was used, theaverage time for surgemes 4, 6, and 7 on a per-trial basis for expertswas 13.34 seconds. This same statistic for intermediates and noviceswere 20.11 and 16.48 seconds, respectively. It thus can be concludedthat choosing to pull the suture in two steps was less time-efficient.Additionally, it can be shown that by choosing to pull the suture to theright across the wound with the right instrument, intermediate andnovice surgeons place undue stress on the tissue that ought to beavoided.

Furthermore, different analytical performance metrics, and time andnumber of motions, can also reveal differences between the threeexpertise level groups. The expert group can show an average of 56.2seconds to complete the task, while intermediates can use an average of77.4 seconds, and novices can complete the task in an average of 82.5seconds. Thus, there is a correlation between time and the number ofsurgemes used in a trial. The average number of surgemes used tocomplete the task were 19, 21, and 20 for experts, intermediates, andnovices, respectively.

By decomposing the time spent per surgeme, observations can be made,such as: (1) experts performed certain surgemes more efficiently thannovices, and (2) experts did not use certain surgemes. FIG. 6illustrates an embodiment in which the time for various surgeme motionsis analyzed. For example, less experienced surgeons typically spent moretime positioning and inserting the needle (surgeme motions 2 and 3,respectively) than experts, particularly to guide the needle tip toemerge through the marked exit point. In one case, manual analysisrevealed that experts spent a per-trial average of 28.04 seconds usingmotions 2 and 3 collectively, intermediates 48.51 seconds, and novices56.59 seconds. As shown in FIG. 6, another indicator of skill was thatexperts hardly used intermediate positioning surgemes, such as motion 5(move to center with right hand), motion 7 (pulling suture with righthand), and motion 8 (orienting the needle with both tools), which areshown by the bottom bars associated with each surgeme in FIG. 6. Whenretrieving the needle from the starting position and when handing theneedle from one tool to the other between suture throws, expert surgeonswere able to grasp the needle in an orientation that did not needreadjusting (i.e., no surgeme motion 8 was indicated for any expert).Intermediates used this two hand orienting motion surgeme twelve timesand required fewer motions to complete a task more quickly than surgeonswith even less skill. Such economy of motion is often subjectivelygauged for surgical skill evaluation, and it is now objectively shown inaccordance with the analysis embodiment illustrated in FIG. 6.

FIG. 7 illustrates an example embodiment analysis of isolated surgemeclassification systems that have been correctly identified. FIG. 7 setsforth eight surgemes, and how they were classified, and how thatclassification compared to training classifications. Reading across therows indicates how many times each surgeme motion was correctlyrecognized and how many times it was mistaken for another skill level.For example, expert surgeme 1 was correctly recognized 8 times andmistaken for intermediate 2 times and novice 2 times. In particular,with respect to surgeme 1, the expert level for surgeme 1 was correctlyclassified as an expert level 50% of the time, incorrectly classified asan intermediate level 28% of the time, and incorrectly classified as anovice level 22% of the time. Similarly, the intermediate level forsurgeme 1 was correctly classified as an intermediate level 67% of thetime, incorrectly classified as an expert level 33% of the time, andincorrectly classified as a novice level 0% of the time. Finally, thenovice level for surgeme 1 was correctly classified as a novice level69% of the time, incorrectly classified as an expert level 31% of thetime, and incorrectly classified as an intermediate level 0% of thetime.

Note that in FIG. 7, there are no models for surgeme motion 5, 7, and 8of an expert, and no models for surgeme motion 4 of an intermediate,because in this example, these surgeme motions were never used by theseexpertise groups. In the example in FIG. 7, there were higherrecognition rates for surgemes where experts performed more efficientlythan novices (surgemes 2, 3, 4) than surgemes that experts did not use(surgemes 5, 7, 8). For the surgemes that experts did not use,intermediates and novices were commonly misclassified with each other,suggesting that they performed these surgemes very similarly. Surgemes 1(66.8% overall; 67% expert; 75% intermediate; 50% novice) and 6 (66.8%overall; 65% expert; 92% intermediate; 50% novice) were difficult toclassify correctly, indicating that certain surgemes are not asdiscriminative of skill as others.

As an additional example of an analysis embodiment, the left sideportion of FIG. 12 illustrates the Cartesian positions of the right handof an expert performing a four-throw suturing task, and the right sideportion of FIG. 12 illustrates the Cartesian positions of the right handof a novice performing the same four-throw suturing task. Various colorsand/or marks along the position lines may be associated with the varioussurgemes each surgeon used during the task. This figure graphicallyillustrates the many differences in movement between a surgeon with anexpert skill level and a surgeon with a novice skill level.

Teaching

FIG. 11 illustrates a method based on the information learned by thequantifying skill application 130 of teaching a user how to perform asurgical task with more proficiency, according to one embodiment. In1105, information about how an expert surgeon performs a procedure ortask (e.g., at the surgeme or dexeme level) is learned by comparingmodule 915. In 1110, the movement of the expert surgeon is modeled usingmodeling module 905. In 1115, a user is taught, using the teachingmodule 920, the movements of an expert surgeon using the modeledmovements found at the expert surgeon level. For example, the user maybe shown how his or her movements compare with an expert's movements byviewing analysis data as illustrated by the various embodimentsdescribed herein. In another embodiment, either a single expert'smotions or a composite of expert motions may be “played back” (with orwithout associated video) via a powered master manipulator, so that anovice may lightly grasp the manipulator and follow along tokinesthetically experience how the expert moves. Similarly, a simulatedmotion of an expert's tool can be displayed in the surgeon's console toallow the novice to follow along by moving a simulated or real tool tomimic the expert's tool motion. If one or more surgemes or dexemes areidentified as particularly difficult to learn, such surgemes or dexemescan be repeatedly played back to the novice and or monitored as thenovice practices the movements until a skill level assessment comparableto the expert's is achieved. And, novice surgeons are motivated toachieve assessment level scores comparable to an expert's. Particulartasks, surgemes, and/or dexemes can be identified for each trainee topractice and master, and the analysis features in accordance withaspects of this invention allow the trainee to quickly assessperformance.

CONCLUSION

While various embodiments of the present invention have been describedabove, it should be understood that they have been presented by way ofexample, and not limitation. It will be apparent to persons skilled inthe relevant art(s) that various changes in form and detail can be madetherein without departing from the spirit and scope of the presentinvention. Thus, the present invention should not be limited by any ofthe above-described exemplary embodiments.

In addition, it should be understood that the figures described above,which highlight the functionality and advantages of the presentinvention, are presented for example purposes only. The architecture ofthe present invention is sufficiently flexible and configurable, suchthat it may be utilized in ways other than that shown in the figures.

Further, the purpose of the Abstract of the Disclosure is to enable theU.S. Patent and Trademark Office and the public generally, andespecially the scientists, engineers and practitioners in the art whoare not familiar with patent or legal terms or phraseology, to determinequickly from a cursory inspection the nature and essence of thetechnical disclosure of the application. The Abstract of the Disclosureis not intended to be limiting as to the scope of the present inventionin any way.

Finally, it is the applicant's intent that only claims that include theexpress language “means for” or “step for” be interpreted under 35U.S.C. 112, paragraph 6. Claims that do not expressly include the phrase“means for” or “step for” are not to be interpreted under 35 U.S.C. 112,paragraph 6.

1. A system for quantifying clinical skill of at least one user,comprising: at least one application operable on at least one computer,the at least one application configured for: collecting data relating toat least one surgical task done by at least one user using at least onesurgical device; comparing the data for the at least one surgical taskto other data for at least one other similar surgical task; quantifyingthe clinical skill of the at least one user based on the comparing ofthe data for the at least one surgical task to the other data for the atleast one other similar surgical task; outputting the clinical skill ofthe at least one user.
 2. The system of claim 1, wherein the surgicaldevice is a surgical robot.
 3. The system of claim 1, wherein the datais video data, motion data, or any combination thereof.
 4. The system ofclaim 1, wherein the at least one application is further configured for:determining at least one expert user based on the comparing; modeling atleast one movement of the at least one expert; teaching at least onenovice user based on the at least one modeled movement of the at leastone expert.
 5. The system of claim 1, wherein the at least oneapplication is further configured for: annotating the at least onesurgical task as being at a novice level, and intermediate level, or anexpert level.
 6. The system of claim 4, wherein the teaching can takeplace without any human supervisor.
 7. The system of claim 1, whereinthe level of clinical expertise of the at least one user isdistinguished using comparisons of various underlying models.
 8. Thesystem of claim 1, where any task where skill is developed underphysical movement can be quantified.
 9. The system of claim 4, whereinthe teaching further comprises: guiding at least one movement of the atleast one novice user based on the at least one modeled movement of theat least one expert.
 10. The system of claim 1, wherein the at least oneapplication is further configured for: collecting data indicating how atleast one skilled user performs at least one surgical task; collectingdata indicating how at least one other user performs the at least onesurgical task; and comparing the collected data for the at least oneskilled user to the collected data for the at least one other user todetermine the clinical skill level of the at least one other user;outputting the clinical skill level of the at least one other user. 11.The system of claim 10, wherein the at least one surgical task is: atleast one surgical trial; at least one surgeme of the at least onesurgical trial; or at least one dexeme of the at least one surgeme. 12.A method for quantifying clinical skill of at least one user,comprising: collecting data to be stored in at least one database, thedata relating to at least one surgical task done by at least one userusing at least one surgical device; comparing the data for the at leastone surgical task to other data for at least one other similar surgicaltask using at least one comparing module; quantifying the clinical skillof the at least one user, using the at least one comparing module, basedon the comparing of the data for the at least one surgical task to theother data for the at least one other similar surgical task; outputtingthe clinical skill of the at least one user to at least one userinterface.
 13. The method of claim 12, wherein the surgical device is asurgical robot.
 14. The method of claim 12, wherein the data is videodata, motion data, or any combination thereof.
 15. The method of claim12, further comprising: determining, utilizing the at least onecomparing module, at least one expert user based on the comparing;modeling, using the at least one modeling module, at least one movementof the at least one expert; teaching, using the at least one teachingmodule, at least one novice user based on the at least one modeledmovement of the at least one expert.
 16. The method of claim 12, furthercomprising: annotating, using the at least one comparing module, the atleast one surgical task as being at a novice level, and intermediatelevel, or an expert level.
 17. The method of claim 15, wherein theteaching can take place without any human supervisor.
 18. The method ofclaim 12, wherein the level of clinical expertise of the at least oneuser is distinguished using comparisons of various underlying models.19. The method of claim 12, where any task where skill is developedunder physical movement can be quantified.
 20. The method of claim 15,wherein the teaching further comprises: guiding, using the at least oneteaching module, at least one movement of the at least one novice userbased on the at least one modeled movement of the at least one expert.21. The method of claim 12, further comprising: collecting, using the atleast one user interface, data indicating how at least one skilled userperforms at least one surgical task; collecting, using the at least oneuser interface, data indicating how at least one other user performs theat least one surgical task; and comparing, using the at least onecomparing module, the collected data for the at least one skilled userto the collected data for the at least one other user to determine theclinical skill level of the at least one other user; outputting, usingthe at least one user interface, the clinical skill level of the atleast one other user.
 22. The method of claim 21, wherein the at leastone surgical task is: at least one surgical trial; at least one surgemeof the at least one surgical trial; or at least one dexeme of the atleast one surgeme.